Mini-Minimax Uncertainty Quantification for Emulators∗

Consider approximating a “black box” function f by an emulator f̂ based on n noiseless observations of f . Let w be a point in the domain of f . How big might the error |f̂(w) − f(w)| be? If f could be arbitrarily rough, this error could be arbitrarily large: we need some constraint on f besides the data. Suppose f is Lipschitz with a known constant. We find a lower bound on the number of observations required to ensure that for the best emulator f̂ based on the n data, |f̂(w)− f(w)| ≤ ε. But in general, we will not know whether f is Lipschitz, much less know its Lipschitz constant. Assume optimistically that f is Lipschitz-continuous with the smallest constant consistent with the n data. We find the maximum (over such regular f) of |f̂(w)− f(w)| for the best possible emulator f̂ ; we call this the “mini-minimax uncertainty” at w. In reality, f might not be Lipschitz or—if it is—it might not attain its Lipschitz constant on the data. Hence, the mini-minimax uncertainty at w could be much smaller than |f̂(w)− f(w)|. But if the mini-minimax uncertainty is large, then— even if f satisfies the optimistic regularity assumption—|f̂(w) − f(w)| could be large, no matter how cleverly we choose f̂ . For the Community Atmosphere Model, the maximum (over w) of the mini-minimax uncertainty based on a set of 1154 observations of f is no smaller than it would be for a single observation of f at the centroid of the 21-dimensional parameter space. We also find lower confidence bounds for quantiles of the mini-minimax uncertainty and its mean over the domain of f . For the Community Atmosphere Model, these lower confidence bounds are an appreciable fraction of the maximum. To know that the emulator estimates f accurately would require evidence that f is typically more regular than it is across the n sample values.